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SUMMARY 


This  research  is  devoted  to  investigating  how  Bayesian  statistical 
procedures  might  be  used  to  improve  the  design  of  operational  tests  being 
conducted  by  the  U.S.  Army  Operational  Test  and  E-. aluation  Agency.  The 
specific  aspect  of  the  design  process  which  is  of  concern  is  the  calcula- 
tion of  required  sample  sizes.  Basically,  three  changes  are  suggested  in 
the  methods  currently  being  employed. 

First,  it  is  shown  that  the  problem  can  be  reformulated  in  a manner 
which  is  believed  to  be  more  closely  aligned  with  the  objectives  of  opera- 
tional testing,  in  so  doing,  it  is  possible  to  capitalize  on  the  compara- 
tive nature  of  the  testing. 

The  problem  is  then  analyzed  using  Bayes'  theorem  and  Bayesian 
inference  techniques.  It  is  felt  that  the  application  of  Bayes'  theorem 
can  provide  for  a more  efficient  use  of  information  available  to  test 
design  personnel  and  that  this  may  result  in  a reduction  in  required 
sample  sizes  when  compared  to  methods  presently  being  utilized.  Formulas 
are  then  derived  for  calculating  the  sample  size  required  to  reduce  the 
expected  value  of  selected  measures  of  tightness  of  the  posterior  dis- 
tribution. 

Finally,  a method  is  proposed  for  utilizing  these  procedures  in 
the  presence  of  economic  considerations  such  as  budget  constraints  and 
sampling  costs.  This  method  attempts  to  find  the  economically  optimal 
sample  size  by  systematically  comparing  the  cost  of  experimentation  with 
the  value  of  the  information  expected  to  be  obtained  from  that  experimen- 
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CHAPTER  I 

INTRODUCTION 


^ Background 

The  impetus  for  this  'study  was  provided  by  the  interest  of  the 
U.S.  Army  'perational  Test  and  Evaluation  Agency  (OTEA)  to  investigate 
the  possible  application  of  Bayesian  statistical  analysis  and  decision 
theory  to  sample  size  determination  for  operational  testing.  In  order  to 
understand  some  of  the  procedures  discussed  later  in  this  study,  a basic 
knowledge  of  the  nature  of  operational  testing  as  performed  by  OTEA  is 
necessary.  The  purpose  of  operational  testing  is  to  provide  a source  of 
data  from  which  estimates  may  be  developed  as  to  the  military  utility, 
operational  effectiveness  and  operational  suitability  of  new  weapon  sys- 
tems. This  data  is  obtained  through  a sequence  of  three  operational  tests; 
each  test  in  the  sequence  is  completed  and  the  results  analyzed  prior  to 
beginning  the  next  test.  For  ease  of  reference,  these  tests  will  be  re- 
ferred to  as  Operational  Test  I (OT  I) , Operational  Test  II  (OT  II)  and 
Operational  Test  III  (OT  III).  Once  the  data  has  been  collected  and  the 
estimates  developed  an  assessment  is  made  of  the  new  system's  desirability 
as  compared  to  systems  which  are  already  available 

The  overall  assessment  procedure  parallels  closel^^^^at  proposed 
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by  Miller  in  his  book.  Professional  Decision  Making  [lO].  InMially, 
certain  issues  concerning  the  system's  capability  are  identified  for 
further  examination  before  any  assessment  of  the  overall  desirability  is 


made.  These  issues  are  general  in  nature  and  cannot  be  resolved  directly. 
For  example,  a system’s  fire  power  may  be  an  issue  of  interest.  This 
cannot  be  represented  by  any  single,  physically  measurable  quantity. 
Therefore,  it  is  necessary  t''  refine  the  issues  into  a number  of  param- 
eters which  adequately  represent  the  issue  and  which  can  be  measured 
quantitatively.  These  quantities  are  known  in  OTEA  as  measures  of  effec- 
tiveness (MOE) . For  the  example  given  above,  the  MOE  might  include  such 
things  as  percent  of  targets  hit,  mean  miss  distance,  percent  of  fire 
requests  which  are  met,  proportions  of  rounds  requested  fired  and  so  on 
[11] . Once  these  MOE  are  identified,  an  operational  test  is  designed  to 
provide  for  a side-by-side  comparison  of  the  competing  systems  with  re- 
spect to  each  MOE. 

Given  a fixed  test  design,  the  sample  size  problem  becomes  one  of 
determining  the  minimum  number  of  replicates  required  for  each  set  of  ex- 
perimental conditions  in  order  to  produce  sufficient  sample  infonnation 
upon  which  to  base  statistically  valid  inferences.  This  problem  can  be- 
come quite  complex  since  a single  operational  test  may  involve  as  many 
as  a hundred  MOE. 

An  approach  which  has  been  recommended  to  reduce  the  computational 
burden  is  to  rank-order  the  MOE  based  on  their  relative  importance  and 
then  to  calculate  the  sample  size  requirements  using  only  two  or  three  of 
the  more  important  MOE.  These  calculations  are  presently  based  on  class- 
ical statistical  procedures  [11]. 
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Objectives  of  Research 

In  re\iewing  these  procedures,  two  areas  of  oossible  improvement 
were  identified.  The  first  is  concerned  with  making  efficient  use  of 
all  available  data.  As  noted  earlier,  the  operational  testing  program 
is  sequential  in  nature  and,  many  times,  the  same  measure  of  effective- 
ness may  be  examined  in  more  than  one  test.  When  this  occurs,  the  data 
from  the  previous  test  is  sometimes  used  in  the  desi gn  of  the  subsequent 
test  in  that  it  serves  as  a basis  for  the  formulation  of  hypotheses  and 
as  a source  of  variance  estimates  for  sample  size  calculations.  This 
data  is  not,  however,  being  combined  with  the  data  obtained  during  later 
tests  in  the  final  statistical  analysis.  By  not  doing  this,  it  is  felt 
that  valuable  information  is  being  wasted.  In  fact,  it  is  believed  that, 
if  this  information  were  used  to  its  fullest  extent,  a reduction  in  the 
required  sample  size  would  be  possible.  One  method  of  combining  prior 
information  with  sample  results  is  provided  by  Bayes*  theorem.  The  next 
chapter  is  devoted  to  investigating  how  Bayes*  cheorem  might  be  applied 
in  the  operational  testing  environment  and  what  effect  this  would  have  on 
the  calculation  of  required  sample  sizes. 

The  second  area  identified  for  possible  improvement  is  concerned 
with  the  economics  involved  in  experimentation.  Presently  the  costs 
associated  with  proposed  experiments  are  .not  directly  considered  in 
sample  size  calculations.  Additionally,  there  is  no  evidence  of  a quan- 
titative assessment  of  the  expected  value  of  the  sample  information  to 
be  obtained  from  a particular  experiment.  Considering  this,  it  is  doubt- 
ful that  the  money  available  for  testing  is  being  allocated  to  the  various 
experiments  in  an  optimal  fashion.  It  is  felt  that  this  problem  might 
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best  be  analyzed  using  the  concepts  of  Bayesian  statistical  decision 
theory.  In  this  manner,  the  economics  of  testing  could  be  considered 
explicitly  and  the  economically  optimal  sample  size  could  be  determined 
for  each  experiment.  In  Chapter  III,  the  application  of  these  ideas  to 
operational  testing  will  be  examined. 

Fundamentals  of  Bayesian  Analysis 
Because  OTEA  is  currently  using  classical  statistical  methods,  the 
discussion  presented  here  will  be  comparative  in  nature.  That  is,  the 
Bayesian  ideas  will  be  contrasted  with  classical  statistical  ideas,  and 
similarities  and  differences  highlighted.  From  the  outset,  there  are 
some  fundamental  conceptual  differences  requiring  discussion.  Consider 
the  situation  in  which  a particular  data-generating  process  may  be  modeled 
by  the  normal  process  with  unknown  mean  and  variance.  Then  the  probabil- 
ity density  function  associated  with  such  a process  is  the  normal  density 

2 

with  mean,  jj,,  and  variance,  a • The  classical  statistician  would  view 

these  parameters  as  unknown  constants.  He  might  decide  to  estimate  them 

by  taking  a sample  from  the  data-generating  process  (or  an  appropriate 

— 2 

model  thereof)  and  use  the  sample  statics  x and  s as  estimates  of  |i  and 
2 

a , respectively.  If  he  is  interested  in  constructing  a confidence  in- 
terval on  X,  he  could  substitute  these  estimates  into  the  normal  density 
function  making  it  possible  to  compute  the  probability  that  a particular 
observation  would  lie  within  a specified  interval,  i.e.,  P(x^  ^ x ^ X2) 

= p.  This  probability  would  then  be  interpreted  in  the  relative  frequency 
sense.  That  is,  if  a large  number  of  observations  were  taken  it  would  be 
expected  that  x would  lie  on  the  interval  (x^,X2),  "p"  percent  of  the 
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! 

' time  [s].  On  the  other  hand,  the  Bayesian  analyst  would  view  the  unknown 

! 2 

parameters,  u and  a - as  random  variables  (throughout  this  paper,  a 

I r- 

I "tilde"  v^ill  be  used  to  denote  a random  variable).  As  such,  he  would  not 

i, 

4 t 

attempt  initially  to  obtain  a point  estimate  of  these  parameters.  In- 

I 

I stead,  he  would  ascribe  to  them  a probability  distribution.  Prior  to 

^ sampli’ig  from  the  process,  such  a probability  distribution  must  be  con- 

! 

' Strutted  based  on  the  analyst's  prior  beliefs  concerning  the  joint  occur- 

2 

rence  of  jl  and  o • A probability  distribution  constructed  in  this  manner 

~2 

reflects  the  analyst's  subjective  probabilities  on  jl  and  a • It  will  be 

shown  later  how  these  probabilities  may  be  combined  with  sample  infoma- 

t 

* tion  to  produce  new  distributions  on  the  unknown  parameters.  The  concep- 

i 

If 

I tual  differences  discussed  here  play  important  roles  in  interpreting  the 

‘ results  of  an  analysis  [23]. 

' ' As  mentioned  above,  Bayesian  analysis  can  be  used  to  combine  sample 

i 

i information  with  prior  beliefs  in  an  effort  to  develop  a probability  dis- 

tribution for  a random  variable.  This  combination  is  achie\'ad  by  using 

•»/ 

Bayes’  theorem.  For  a continuous  random  variable,  say  9,  Bayes  tu'^orem 
may  be  written  as 


f'Cely) 


f'(9)f(y|e) 

J f'(e)f(yl0)de 

^ -00 


(1-1) 


In  this  notation,  a single  prime  superscript  (')  denotes  a prior  distri- 
bution or  parameter,  a double  prime  (")  denotes  a posterior  distribution 


¥C 

The  notation  used  in  this  study  is  similar  to  that  used  by  Raiffa 
and  Schlaifer  [iS].  At  times  it  can  become  quite  intricate;  therefore,  a 
detailed  explanation  of  this  notation  is  presented  in  Appendix  III. 
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or  parameter  and  no  superscript  designates  a sampling  distribution  or 
parameter.  Therefore,  in  equation  (1-1),  f'(0)  is  the  prior  distribution 
of  0 representing  the  decision  maker's  beliefs  regarding  0 prior  to  samp* 
ling,  f(y|9)  represents  the  likelihood  function  chosen  to  describe  the 
sampling  process  and  f"(0|y)  is  the  posterior  distribution  of  0 which  re- 
flects the  decision  maker's  beliefs  regarding  0 after  the  sample  has  been 
taken  [23],  An  analogous  form  of  the  theorem  may  be  written  for  discrete 
random  variables  by  substituting  probability  mass  functions  for  the  prob- 
ability density  functions  and  a summation  sign  for  the  integral  sign.  A 
derivation  of  Bayes  * theorem  from  conditional  probability  formulas  is 
given  by  Winkler  [23].  In  the  application  of  Bayes'  theorem  the  major 
difficulties  lie  in  the  assessment  of  the  prior  distribution  and  the 
likelihood  function  and,  in  the  continuous  case,  in  the  evaluation  of  the 
integral  appearing  in  the  denominator  of  the  formula.  Suggested  methods 
for  handling  these  difficulties  are  discussed  in  the  next  chapters. 
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CHAPTER  II 

AN  INFERENTIAL  APPROACH 

Bayesian  Inference 

Statistical  inference  is  the  process  of  forming  reasonable 
conclusions  about  some  aspect  of  a random  phenomenon.  For  example,  in 
considering  the  mean,  p,,  of  a normal  distribution,  the  classical  statis- 
tician may  attempt  to  estimate  the  true  value  of  p based  on  sample  infor- 
mation. Alternatively,  he  may  construct  a confidence  interval  on  p of 
the  form  described  in  Chapter  I.  The  Bayesian  statistician  also  makes 
inferential  statements  regarding  p.  These  statements,  however,  are  de- 
veloped in  a different  manner  and  have  different  interpretations  than  their 
classical  counterparts. 

As  pointed  out  in  Chapter  I,  the  Bayesian  considers  p to  be  a ran- 
dom variable  and  assigns  to  it  some  probability  distribution.  Inferential 
statements  concerning  p are  then  based  on  this  distribution.  For  in- 
stance, while  the  classical  statistician  may  estimate  the  true  value  of 
p,  the  Bayesian  may  be  interested  in  an  estimate  of  the  most  likely  value 
of  p and  may  use  as  a point  estimate  the  mode  of  the  distribution  [23]. 

In  making  interval  estimates  of  p,  the  Bayesian  attempts  to  define  an 
interval  [a,b]  such  that  the  probability  that  p will  take  on  values  be- 
tV7een  a ana  b is  some  number  "p.  " 

Thus,  Bayesian  inference  revolves  around  the  distribution  of  the 
unknown  quantity  of  interest.  If  the  analyst's  present  state  of  knowledge 


about  this  unknovm  quantity  is  sufficient  to  develop  a well-defined  prio 
distribution,  then  this  distribution  may  be  used  for  the  analysis.  On 
the  other  hand,  if  the  analyst’s  prior  knowledge  is  vague,  he  may  con- 
sider gathering  sample  infomation  in  order  to  produce  a posterior  distr 
bution  upon  which  to  base  his  i.iferences.  In  this  case,  the  problem  is 
determine  how  much  sample  information  is  necessary  to  produce  a suitable 
posterior  distribution.  !fti7o  possible  approaches  to  solving  this  problem 
are  presented  later  in  this  chapter.  First,  however,  the  two  required 
inputs  to  Bayes'  theorem,  i.e.,  the  likelihood  function  and  the  prior 
distribution,  will  be  discussed  in  the  context  of  operational  testing. 

The  Likelihood  Function 

In  operational  testing,  the  value  of  the  MOE  under  consideration 
may  be  thought  of  as  the  uncertain  state  of  nature  and  may  be  represente 
by  9.  If  y is  a sufficient  statistic  for  a sample  from  the  data-generat 
process,  then  prior  to  sampling,  y is  also  a random  variable.  The  proba 
bility  distribution  of  y is  assumed  to  depend  on  0,  and  the  conditioned 
probability  distribution  of  y given  6 will  be  denoted  by  f(y|0)  and 
called  the  "likelihood"  function.  In  order  to  proceed  with  the  analysis 
it  is  necessary  to  mathematically  describe  this  function.  In  doing  this 
subjective  probability  assessments  could  be  made  for  each  data-generatin 
process  encountered  and  unique  likelihood  functions  constructed.  This 
could  prove  extremely  cumbersome  considering  the  number  of  MOE  involved 
in  a single  operational  test.  A better  approach,  and  one  used  more  fre- 
quently in  practice,  is  to  attempt  to  "fit"  one  of  the  more  common  sta- 
tistical models  to  the  process.  As  pointed  out  by  Winkler  [23],  this 


does  not  eliminate  the  subjectivity  involved  in  assessing  the  likelihood 
function,  although  it  may  make  it  somewhat  less  controversial. 

In  choosing  a likelihood  function  to  be  used  in  this  study,  it  was 
desired  to  select  a function  which  would  realistically  represent  a broad 
class  of  MOE.  The  univariate  normal  with  unknown  mean  and  variance  is 
such  a function.  Its  applicability  to  a wide  range  of  MOE  is  supported 
by  the  fact  that  it  is  currently  being  used  by  OTEA  as  the  basic  model 
for  sample  size  determination  for  both  measurement  and  attribute  data 
[ill.  It  should  be  stressed,  however,  that  this  likelihood  function  should 
not  be  used  indiscriminately  but  only  when  the  decision  maker’s  prior  be- 
liefs concerning  the  data-generating  function  suggest  that  it  v7ould  be  an 
appropriate  model. 


The  Prior  Distribution 

Before  any  operational  testing  is  conducted,  the  prior  distribution 
would  have  to  be  assessed  based  on  the  decision  maker's  prior  notions  con- 
cerning 0 or,  in  the  case  of  a totally  informationless  situation,  would 
have  to  be  represented  by  a diffuse  distribution,  which  will  be  discussed 
later.  After  at  least  one  operational  test  has  been  conducted,  these 
sample  results  might  be  used  in  constructing  the  priors  for  similar  MOE 
in  later  tests  [ll]. 

In  selecting  a prior  distribution,  several  desirable  characteris- 
tics should  be  considered.  First,  and  most  important,  the  distribution 
should  adequately  reflect  the  decision  maker's  prior  beliefs.  Second,  it 
should  be  of  such  a form  as  to  be  mathematically  tractable  when  combined 
with  the  likelihood  function  in  Bayes'  theorem.  And,  finally,  it  would 
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be  desirable  to  have  the  resulting  posterior  distribution  be  of  the  same 
form  as  the  prior  so  as  to  reduce  the  computational  burden  in  sequential 
analysis. 

By  choosing  a prior  distribution  from  the  natural  conjugate  family 
of  distributions  it  is  possible  to  guarantee  that  the  second  and  third 
conditions  above  are  met  (for  a definition  arid  complete  discussion  of 
natural  conjugate  distributions  see  Raiffa  a.id  Schlaifer  [l8j,  Chapter  3). 
l*7hile  the  first  condition  is  not  guaranteed  to  be  satisfied,  it  is  pro- 
vided for  by  the  fact  that  natural  conjugate  families  are  generally  "rich" 
and  through  proper  parameterization  can  be  made  to  represent  a wide  var- 
iety of  distributions. 

The  choice  of  a natural  conjugate  family  is  determined  by  the  form 
of  the  likelihood  function.  Since  the  likelihood  function  used  here  is 
normal  with  unknown  mean  and  variance,  the  appropriate  conjugate  family 
is  the  normal-gamma  distribution  [l8].  Which  member  of  this  family,  if 
any,  will  be  appropriate  depends  on  the  amount  and  substance  of  the  de- 
cision maker's  prior  beliefs  concerning  the  specific  MOE  under  considera- 
tion. Certain  peculiarities  of  operational  testing  will  allow  for  some 
general  statements  to  be  made  concerning  the  decision  maker's  prior  state 
of  knowledge.  First  is  the  requirement,  imposed  by  the  Department  of  the 
Army,  that  operational  testing  be  independent  of  all  other  testing.  This 
severely  limits  the  use  of  any  prior  knowledge  on  0.  For  all  practical 
purposes,  prior  to  OT  I there  exists  a self-imposed,  totally  information- 
less situation.  In  such  cases,  the  prior  information,  or  lack  thereof, 
should  be  represtr.ted  by  a diffuse  prior  distribution. 

In  general,  a diffuse  distribution  need  not  be  limited  in  its  use 
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to  a totally  informationless  situation,  but  may  be  used  whenever  the 
decision  maker's  prior  information  is  diffuse  relative  to  that  which  can 
be  obtained  through  sampling.  As  a rule,  diffuse  distributions  are  char- 
acterized by  large  variances  as  compared  to  that  of  the  datn-generating 
function,  they  are  relatively  flat  in  the  region  where  the  likelihood 
function  takes  on  significant  values,  and  they  are  given  little  weight  ir> 
comparison  to  the  sampling  distribution  when  computing  the  posterior  dis- 
tribution [23].  The  real  objective  in  selecting  a diffuse  distribution 
is  to  choose  one  which  will  have  no  effect  on  the  posterior  distribution 
[23].  With  this  objective  in  mind,  the  actuil  forrri  of  a diffuse  prior 
is  of  little  importance,  in  fact,  it  need  i.ut  even  be  a proper  probability 
distribution  [23].  Tnus,  it  is  only  rat^opal  to  choose  a diffuse  distri- 
bution from  the  family  of  natural  conjugate  distributions,  in  this  case, 
the  normal-gamma  family.  This  family  of  distributions  is  of  tie  follow- 
ing form  [I8], 


c ul  I I I i\  • ''}hn ' (a-m ' ) ^ -%hv'v’  ?rtv’-l 

fj^  ,(n,h|m' ,v' ,n' ,v  ) " e ‘ ' h e h^ 


where  ^ and  K are  random  variables  such  that 

- « < < u' 

h > 0 

and  m',  v',  n*,  v'  are  parameters  such  that 
v' , n' , v'  > 0 

- os  < ra'  < os 

The  above  paramuters  may  be  interpreted  as  previous  sample  results  asso- 
ciated with  some  actual  or  hypothetical  experiment  [231.  Using  this 
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interpretation,  m*  would  be  equivalent  to  the  sample  mean,  v’  would  be 
the  sample  variance,  n*  the  size  of  the  previous  sample  and  v*  equal  to 
n’  minus  one  (usually  called  the  number  of  de^irees  of  freedom  for  v*). 

By  the  properties  of  natural  conjugate  distributions,  it  is  known 
that  the  posterior  distribution  will  be  of  the  same  form,  Raiffa  and 
Schlaifer  [l8]  have  shown  that  the  posterior  parameter  (m",v",n",v'’)  is 
given  by 


II  _ n*n:*  + nm 

m — ; 

n + n 

n"  - n*  + n 

II  _ v*v*  + n'm*^  + w 4-  nm^  - n'm”^ 
V*  + 6(n*)  +V  + 6(n)  - 6(h") 

v"  = v'  + 6(n’)  + V + 6(n)  - 6(n") 


(2-1) 


where  6(y)  is  an  indicator  variable  defined  by 

1 if  Y > ^ 

6(y)  = 

0 if  Y = 0 

and  (m,v,n,v)  is  the  statistic  resulting  from  a sample  of  size  n and  is 
given  by 

EX. 

^ 1 

n 

S(X.-m)^ 

V = n-1 

It  is  shown  in  reference  18,  p.  300  that,  if  n'=v'=0,  the  posterior 
parameter  (m",v",n",v")  equals  the  sampling  statistic  (m,v,n,v).  There- 


fore,  any  normal-gamma  distribution  can  be  used  as  a diffuse  prior  so 
long  as  n'=v'=0. 

Now,  consider  the  case  where  at  least  one  operational  test  has 
already  been  conducted.  This  will  provide  usable  data  on  a number  of  MOE. 
Commonly,  several  of  these  MOE  will  be  subject  to  further  examination  in 
subsequent  operational  tests.  Under  these  circumstances  the  posterior 
Distribution  of  the  earlier  test  may  be  used  as  the  prior  distribution 
for  the  later  test.  This  is  especially  useful  if  the  distributions  in- 
volved are  natural  conjugates  of  the  likelihood  function. 

Even  if  the  MOE  to  be  evaluated  in  later  tests  have  not  been  ex- 
amined previously,  they  may  possess  strong  similarities  to  MOE  which  have. 
In  such  cases,  it  might  be  possible  to  construct  prior  distributions 
based  on  subjective  probability  notions  deriveo  from  data  on  thi  earlier 
MOE. 

In  summary,  if  there  is  no  internally-generated  data  available 
from  which  to  develop  prior  distributions,  a diffuse  normal-gamn.a  prior 
will  be  used.  If  the  available  data  is  of  the  form  of  a previous  sample 
from  the  same  data-generating  process,  then  the  posterior  distribution  of 
the  earlier  test  will  be  used  as  the  prior  for  the  later  test.  Finally, 
if  the  available  data  is  of  the  form  of  a sample  from  a similar  data- 
generating  process,  then  this  posterior  distribution  will  be  used  in  mak- 
ing a subjective  assessment  of  the  prior  distribution  to  be  uc?'’  in  the 


later  test. 
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The  General  Problem 

As  noted  in  Chapter  I,  operational  tests  are  designed  to  provide 
a side-by-side  comparison  of  competing  weapons  systems.  The  overall  ob- 
jective of  this  testing  is  not  so  much  to  estimate  the  performance  char- 
acteristics of  either  of  the  systems,  but  rather  to  make  inferences 
regarding  the  difference  in  these  performance  characteristics.  In  this 
context,  the  observations  may  be  considered  to  be  paired  observations  and 
the  difference  in  the  observations  may  be  viewed  as  a random  variable  with 
its  own  probability  distribution.  Consider,  as  a hypothetical  example, 
the  problem  of  determining  whether  a new  weapons  system  has  a greater 
range  than  that  of  the  existing  system  which  it  has  been  designed  to  re- 
place. Let  the  range  of  the  existing  system  be  denoted  by  and  that  of 
the  new  system  by  X2.  Assume  that  the  prior  information  on  and  X2  is 
such  that  both  may  be  modeled  by  the  normal  process  with  unknown  mean  and 
variance.  Then,  the  difference  In  the  range  of  the  two  systems  is  also  a 
random  variable,  D,  given  by 

D = Xj^  - X2  (2-2) 

From  equation  (2-2) , D is  merely  the  linear  combination  of  two 

independent,  normally  distributed  random  variables,  which  implies  that  D 

is  also  normally  distributed  [8]  with  unknown  mean  and  variance,  say  p 
2 

and  a , respectively.  Using  this  distribution  of  D,  the  family  of  natural 
conjugate  prior  distributions  is  normal-gamma.  The  mathematics  involved 
in  working  v/ith  this  particular  family  of  distributions  can  be  quite  com- 
plex, fortunately,  this  may  not  be  necessary.  I^at  is  of  particular  im- 
portance in  this  testing  is  the  mean  difference,  {I,  between  the  two 


systems.  Since  fj  is  itself  a random  variable,  it  follows  a particular 
probability  distribution.  It  has  been  shown  that  this  distribution  is 
Student's  t distribution  and  can  be  represented  by  the  density  [iS] 

f (il|m,v,n,v)  = fg({l|m,n/v,v)  (2-3) 

Formulated  in  this  manner,  the  problem  becomes  one  of  determining 
the  minimum  sample  size  which  can  be  expected  to  produce  a posterior  dis- 
tribution suitable  for  making  meaningful  probability  statements  regarding 
p.  One  approach  to  solving  this  problem  is  to  identify  some  measure  on 
the  posterior  distribution  which  is  a function  of  the  sample  size,  derive 
the  expected  value  of  this  measure,  equate  this  expected  value  to  some 
desirable  value  and  solve  for  the  sample  size.  Solution  procedures  util- 
izing two  such  measures  have  been  developed  and  are  presented  in  the  next 
two  sections. 

'■I’le  Solution  Using  the  Standard  Deviation 

Given  that  p has  the  density  described  by  equation  (2-3),  then  [l8] 

E(p|m,v,n,v)  = jl  = m , v > 1 

V(p|m,v,n,v)  = |A  = “ > V > 2 

Under  the  prior  distribution  of  p,  the  value  of  jj,'  is  given  by 

V , _ v^  v' 

~ n'  v'-2  ’ 


v'  > 2 


(2-4) 
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fhis  value  may  then  be  used  to  calculate  the  prior  standard  deviation  of 
p,  i.e.js/p^  . If  the  prior  standard  deviation  is  large  relative  to 
p,’,  then  the  prior  distribution  may  not  be  "tight"  enough  to  reach  any 
meaningful  conclusions  about  p.  In  this  situation,  it  may  be  necessary 
to  obtain  additional  information  about  p through  sampling.  The  objective 
of  this  sampling  would  be  to  produce  a value  for  the  posterior  standard 
deviation  which  would  be  small  enough  to  allow  inferences  to  be  made 
about  p. 

Suppose  that  it  is  felt  a posterior  standard  deviation  equal  to 
some  specific  fraction  of  the  prior  standard  deviation  would  be  satisfac- 
tory. Mathematically,  this  relationship  is 

0<SS1.  (2-5) 


Prior  to  sampling,  the  posterior  standard  deviation  is  a random  variable 
and,  therefore,  it  is  necessary  to  think  in  terms  of  its  expected  value. 
This  has  been  shown  to  be  [iS] 


(2-5a) 


Im’.V.n'.v'in.v)  v’-D- (l/l/2 


In  deriving  the  above  equation,  it  was  necessary  to  use  Stirling's  second 
approximation  for  the  following  two  values 

/ , (2-5b) 

(1/2  v'-D;  = (2„)1/2(i/2  v'-l)-Kl/(12(l/2  v'-l))) 

(2-5c) 

(1/2  »”-l)l  = (2„)1/^i/2  v"-l).(l/(12(l/2  v"-l)))  , 


It  has  been  suggested  [l8,p.  308]  that,  if  Stirling's  first  approximation 
were  used,  the  second  term  in  the  exponent  of  "e,"  in  both  equations 
(2-5b)  and  (2-5c),  would  be  omitted  and  the  expected  value  would  then 
become 


e[  \/jS"  ‘|m‘  ,v*  ,n ' ,v'  ;n,v]  = \/(n '/n‘*)p,'  ‘ . (2-6) 

Thus,  in  using  Stirling's  first  approximation,  some  information  is  sacri- 
ficed in  order  to  obtain  a less  complex  mathematical  expression  for  the 
expected  value  of  the  posterior  standard  deviation.  The  question  then 
becomes  one  of  how  much  information  is  lost  and  is  this  loss  justified. 

A problem  similar  to  this  is  encountered  in  classical  statistics 
when  attempting  to  arrive  at  an  unbiased  estimator  of  the  standard  devia- 
tion. One  solution  to  this  problem  is  to  multiply  the  sample  standard 
deviation  by  an  appropriate  correction  factor.  Gurland  and  Tripathi  have 
shown  that  this  correction  factor  approaches  one  as  the  size  of  the  sample 
increases.  ^ In  fact,  for  a sample  of  size  20,  the  correction  factor  is 
1.0132,  implying  that  the  sample  standard  deviation  varies  from  the  un- 
biased estimator  of  the  population  standard  deviation  by  only  slightly 
more  than  one  percent. 

It  is  felt  that  the  problem  of  approximating  equation  (2-5a)  by 
equation  (2-6)  may  be  viewed  in  a similar  fashion  where  the  exponential 
term  in  equation  (2-5a)  is  analogous  to  the  correction  factor  discussed 
above.  Therefore,  the  percent  e^ror  induced  by  the  approximation  can  be 

Gurland,  J.,  and  Tripathi,  R.  C.,  "A  Simple  Approximation  for 
Unbiased  Estimation  of  the  Standard  Deviation,"  The  American  Statistician, 
Vol.  25,  1971,  pp.  30-32. 
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expressed  as 

% error  = 1 - v’-l)  - (1/1/2  v"-l)). 

Note  that,  using  the  relationships  [l8] 

V = V + V + 1 

and 

V = n - 1 , 

equation  (2-6a)  may  be  rewritten  as 

% error  = 1 - ' (lA/2(v'+n)-l)) 

^ ^ _ ^-6/8((l/v’-2)  - (l/v'+n-2))^ 

This  equation  was  used  to  calculate  the  percent  error  of  approximation 
for  selected  values  of  n and  v',  and  the  results  are  presented  in  Table  1. 
Notice  that,  for  a given  value  of  v',  the  accuracy  of  the  approximation 
decreases  to  a limit  as  the  sample  size  approaches  infinity.  This  phe- 
nomenon makes  it  possible  to  establish  an  upper  bound  on  the  approxima- 
tion error  for  any  given  value  of  v'.  For  the  values  of  v'  considered  in 
Table  1,  this  upper  bound  is  shown  in  the  last  row  of  the  table.  From 
this  information  it  can  be  seen  that  the  approximation  is  reasonably  accu- 
rate for  values  of  v'  greater  uhan  or  equal  to  35  regardless  of  the  size 
of  the  sample.  For  values  of  v'  less  than  35,  the  decision  about  whether 
or  not  to  use  the  approximation  would  have  to  be  made  based  on  a compari- 
son of  the  percent  error  induced  in  the  calculations  versus  the  desired 
accuracy  of  the  results. 
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Table  1.  Percent  Error  in  Approximating  the  Expected 
Posterior  Standard  Deviation 


V 

5 

10 

15 

20 

25 

30 

35 

40 

45 

50 

5 

14 

4 

2 

1 

1 

0 

0 

0 

0 

0 

10 

17 

5 

2 

1 

1 

1 

1 

0 

0 

0 

15 

19 

6 

3 

2 

1 

1 

1 

1 

0 

0 

20 

19 

6 

3 

2 

2 

1 

1 

1 

1 

0 

25 

20 

7 

4 

2 

2 

1 

1 

1 

1 

1 

30 

20 

7 

4 

3 

2 

1 

1 

1 

1 

1 

35 

20 

7 

4 

3 

2 

1 

1 

1 

1 

1 

40 

21 

8 

4 

3 

2 

2 

1 

1 

1 

1 

45 

21 

8 

4 

3 

2 

2 

1 

1 

1 

1 

CQ 

22 

9 

6 

4 

3 

3 

2 

2 

2 

2 

The  procedures  developed  in  the  remainder  of  this  study  utilize 
the  approximate  expression  for  the  expected  value  of  the  posterior  stand- 
ard deviation.  It  is  be^eved  that  this  approximation  will  be  acceptable 
in  the  design  of  a large  percentage  of  operational  tests.  When  it  is  not 
acceptable,  the  methodology  presented  in  this  study  might  still  be  appli- 
cable, although  the  specific  results  would  not  be.  For  example,  it  may 
be  possible  to  use  the  exact  expression  given  by  equation  (2-5a).  Al- 
though this  equation  cannot  be  solved  explicitly  for  n,  it  can  be  solved 
iteratively.  In  the  iterative  solution,  it  is  suggested  that  the  analyst 
use  equation  (2-6)  to  calculate  a first  approximation  for  n. 

Returning  to  the  development  of  the  methodology,  if  equation  (2-6) 
is  used  in  place  of  the  posterior  standard  deviation  in  equation  (2-5), 
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then 


N/(n’/n"){l'  = s\/^T^  . 


Squaring  both  sides 


n ' V j _ „2  V I 

^ (1  - s n 


Using  equation  (2-1)  , this  may  now  be  solved  for  n giving 

n = - l)n'  , 0 < s s 1 . (2-7) 

■ s 

In  essence,  the  above  equation  states  that  a sample  of  size  n can  be  ex- 
pected to  reduce  the  prior  standard  deviation  of  jl  by  some  factor  s. 

This  has  a certain  amount  of  intuitive  appeal.  Notice  that,  if 
s=l,  indicating  that  the  prior  standard  deviation  is  satisfactory,  the 
sample  size  is  zero  regardless  of  the  value  of  n'.  Additionally,  if  n' 
is  interpreted  to  be  the  "weight"  assigned  the  prior  distribution,  as 
suggested  by  Winkler  [23],  then  as  the  prior  distribution  is  given  more 
weight  in  the  analysis,  the  sample  size  increases.  This  is  reasonable 
since  the  weight  the  analyst  assigns  to  the  prior  dis*-ribution  reflects 
his  confidence  in  that  distribution.  Thus,  if  he  has  a great  deal  of 
confidence  in  the  prior,  it  would  take  a large  amount  of  sample  informa- 
tion to  significantly  alter  his  beliefs. 

The  Solution  Using  a Bayesian  Interval 
Suppose  that  the  decision  maker  would  prefer  to  use  some  other 
measure  of  the  posterior  distribution.  A reasonable  measure  would  be  a 
Bayesian  pi'ediction  interval.  The  development  of  the  solution  procedure 
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is  similar  to  the  previous  section.  A Bayesian  prediction  interval  is  an 
interval  having  a stated  probability,  e.g.,  (l-of)  > containing  the  vari- 
able of  interest.  In  Figure  1,  jT”  is  the  mean  of  the  posterior  distribu- 
tion, a is  the  lower  prediction  limit,  b is  the  upper  prediction  limit, 
and  the  shaded  area  represents  the  probability  that  a 2 s b. 


Figure  1.  Generalized  Bayesian  Interval  on  p" 

Now,  assuming  that  the  interval  is  centered  on  p",  then  the  dis- 
tance, d",  from  a to  b is  given  by  [S] 

Prior  to  sampling,  the  decision  maker  would  be  interested  in  the  expected 
value  of  d",  which  can  be  expressed  as 

EW")  = '>■ 

Using  equation  (2-6)  this  becomes 


Squaring  both  sides  gives 


This  equation  can  then  be  solved  for  n yielding 


(2-9) 


The  above  equation  parallels  closely  a result  obtained  by  Cordova  [6] 
where  he  shows  that  the  minimuir.  sample  size  required  to  establish  a Bay- 
esian interval  of  expected  v/idth  k about  the  mean  of  a sampling  distribu- 

2 

tion  when  the  variance  of  that  distribution,  a , is  known  is 


2 

He  goes  on  to  demonstrate  that  the  quantity  is  equivalent  to 

the  classical  solution  to  the  same  problem. 

Note  that  (2-9)  cannot  be  solved  explicitly  for  n.  It  is  suggested  , 
that  it  be  solved  by  the  trial  and  error  method.  A good  first  approxima- 
tion for  n may  be  found  bv  using  place  of  v"  solving  for 

n. 

Illustrating  the  Procedures 

In  this  section,  an  example  will  be  given  of  how  each  of  the  solu- 
tion procedures  may  be  applied  in  a realistic  situation.  It  was  decided 
! uju  the  procedures  in  the  context  of  an  actual  operational  test.  The 
test  selected  was  OT  II  for  the  Lightweight  Company  Mortar  System  (LWCMS) . 


The  LWCMS  is  being  considered  as  a replacement  for  the  M29  Slmm  mortar 
currently  being  used  by  the  Army.  The  purpose  of  the  test  is  to  provide 
comparative  data  on  the  two  types  of  mortars  for  assessing  the  relative 
operational  performance  and  military  utility  of  the  LWCMS  [l4].  One  of 
the  MOE  under  consideration  in  this  test  is  the  time  required  for  an  indi- 
vidual to  complete  the  gunner’s  examination.  The  gunner's  examination  is 
a practical  test  designed  to  measure  how  quickly  an  ii. dividual  can  per- 
form certain  essential  operations  in  preparing  a mortar  to  fire. 

This  MOE  was  previously  examined  during  OT  I.  In  that  test,  14 
individuals  were  given  the  gunner's  exam  using  the  Slmm  mortar.  They 
were  then  presented  with  tv;o  weeks  of  instruction  on  the  LWCMS,  after 
which  CbAy  once  more  took  the  gunner's  exam,  this  time  using  the  LWCMS. 

The  results  of  this  test  are  contained  in  Appendix  I.  The  format  for  the 
experiment  in  OT  II  is  the  same.  The  sample  size  problem  is  to  determine 
the  number  of  individuals  to  be  used  in  that  experiment.  The  first  solu- 
tion procedure  to  be  illustrated  will  use  the  standard  deviation  as  the 
measure  on  the  posterior  distribution. 

The  initial  step  in  the  procedure  is  to  determine  the  value  of  the 
prior  standard  deviation  of  For  notational  purposes,  the  sample  data 
relevant  to  the  Slmm  mortar  will  be  denoted  by  i = 1,2,  . . .,14 

and  that  associated  with  the  LWCMS  by  ^2^,  i = 1,2,  . . .,  14.  To  compute 
the  value  of  \/^  it  is  necessary  to  know  n',  v',  and  v'  of  the  prior 
distribution.  Since  this  MOE  was  examined  previously,  the  prior  distri- 
bution for  OT  II  may  be  equated  to  the  posterior  distribution  of  OT  I. 

As  pointed  out  in  Chapter  II,  prior  to  OT  I there  is  usually  no  internally 
generated  data  available;  therefore,  a diffuse  prior  distribution  is  ap- 
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propriate.  Thus,  the  posterior  distributions  associated  with  OT  I are 
based  solely  on  sample  l-iformation.  Considering  this,  the  posterior 
parameters  relative  to  OT  I are  given  by  [l8,  p.  298] 


S.D. 

II  _ 1 1 

m = m 

n 


, i:(D  -m)' 

■y  — y S= 


n"-l 


where 


n = n 


V = n -1  = n-1 


D.  = X, . - X„.  . 
X lx  2x 


Using  the  data  from  Appendix  I,  the  follov7ing  values  were  calculated: 
n = 14 
n"  = 14 
ra"  = 17.6  sec 
v"  = 2040.5  sec^ 
v"  = 13  . 

The  above  values  may  now  be  used  as  the  parameters  of  the  prior  distribu- 
tion relative  to  OT  II. 

At  this  point  it  is  possible  to  calculate  an  upper  bound  for  the 
error  induced  by  using  equation  (2-6)  as  an  approximation  for  the  expected 
standard  deviation.  With  v'  equal  to  13,  this  upper  bound  is  six  percent. 
If  it  is  felt  that  a possible  error  of  this  magnitude  is  critical,  then 
the  approximation  should  not  be  used.  It  will  be  assumed  here  that  such 
an  error  is  not  crxtical.  The  next  step,  then,  is  to  calculate  the  value 
of  the  prior  variance  of  jj.  Using  equation  (2-4) 
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V,  /204Q.5V  13  \ 
^ ~K  14  A 13-2/ 


fb*  = 172.25  sec 


This  produces  a prior  standard  deviation  of 


\/^  = 13. 12  sec  . 


i 


The  fact  that  this  MOE  is  again  being  considered  in  OT  II  implies  the 
above  standard  deviation  is  too  large  to  formulate  meaningful  conclusions 
regarding  p.  What  specific  value  of  the  posterior  standard  deviation 
would  be  acceptable  is  something  which  must  be  determined  by  the  OTEA 
test  designers.  To  assist  in  this  decision,  Table  2 depicts  the  sample 
sizes  required  to  produce  various  expected  values  for  the  posterior 
standard  deviation. 


Table  2,  Required  Sample  Sizes  for  Values  of  the  Expected 
Posterior  Standard  Deviation  (in  seconds) 


E(x/^  ) 12.0  11.0  10.0  9.0  8.0  7-.0  6.0  5.0  4.0  3.0  2.0  1.0 

n 3 6 11  16  24  36  53  83  137  254  589  2396 


The  values  of  n were  found  by  using  equation  (2-7)  with 


13.12  • 
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All  that  remains  is  for  the  analyst  to  select  the  desirable  value  for  the 
expected  posterior  standard  deviation  and  obtain  the  required  sample  size 
from  Table  2. 

Now  consider  the  solution  procedure  which  uses  a Bayesian  interval 
on  the  posterior  distribution  as  a measure.  Based  on  the  prior  distribu- 
tion, the  length  of  an  interval,  centered  on  the  mean,  containing  90%  of 
the  probability  is  given  by 


d'  = 2t  /„ 

q'/2.,v 

= 2(1.761) (13.12) 

= 46.21  sec  i 

Suppose  that  it  is  desired  to  have  the  expected  width  of  the  Bayesian 
interval,  with  respect  to  the  posterior  distribution,  be  equal  to 
E(d")  = 20.00  sec  , 

then 

E(d")^  = 400.00  sec^  . 

Using  equation  (2-9) 


(25  „)^(172.25) 

liiizV (IAN  _ 

400 


To  obtain  a first  approximation  for  n 
giving 


14  . 

Z is  substituted  for  t m 
.05  .05,v 


= (w) . M 


n = 51.26  . 

Rounding  this  up  to  the  next  greatest  integer  gives  an  initial  value  for 
n of  52.  Using  this  sample  size,  n"  would  equal  66,  with  the  correspond- 
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ing  value  of  t being  1.6686.  Substituting  this  value  in  equation 

(2-9)  and  solving  for  n gives 


n 


Mi6686l!p^  (14)  14 


n = 53.14  . 

From  this  result  it  appears  that  the  optimal  n will  lie  somewhere  between 

52  and  54.  Setting  n equal  to  53  and  using  the  appropriate  value  for 

t /o  II  gives 
0'/2,v 


n 


4(1.6683)^(172.25) 

400 


(14)  - 14 


n = 53.12  . 

Therefore,  a sample  of  size  54  would  reduce  the  expected  width  of  a 90% 
Bayesian  prediction  interval  to  20. 

The  procedures  developed  in  this  chapter  have  not  considered  any 
monetary  constraints  associated  with  the  cost  of  sampling..  In  reality, 
however,  such  constraints  play  an  important  role  in  the  determination  of 
the  optimal  sample  size.  For  example,  it  may  be  desirable  to  hdve  an 
expected  posterior  standard  deviation  equal  to  one;  however,  the  cost  of 
having  2396  individuals  involved  in  the  experiment  may  be  prohibitive. 
Therefore,  it  is  necessary  to  achieve  some  balance  between  the  E(  ) 
and  the  cost  of  sampling.  This  is  the  subject  of  the  next  chapter. 


28 


CHAPTER  III 

A DECISION  THEORETIC  APPROACH 

Introduction 

The  problem  which  was  examined  in  Chapter  II  will  now  be  expanded 
to  include  economic  considerations.  The  objective  of  this  chapter  will 
be  to  develop  a procedure  for  determining  the  economically  optimal  sample 
size  in  the  presence  of  monetary  constraints.  Initially,  it  appears  that 
this  problem  could  best  be  handled  using  Bayesian  preposterior  analysis. 

In  Bayesian  preposterior  analysis,  the  decision  maker  examines  each  ex- 
periment available  to  him  in  an  effort  to  determine  if  the  expected  value 
of  the  sample  information  is  sufficient  to  justify  the  expected  cost  of 
the  experiment.  If  more  than  one  experiment  satisfies  this  criterion, 
he  then  chooses  the  experiment  which  gives  him  the  greatest  expected  net 
gain  from  sampling,  where  the  expected  net  gain  from  sampling  is  equal  to 
the  expected  value  of  sample  information  minus  the  expected  cost  of  samp- 
ling [l8]. 

The  problem  in  this  approach,  however,  is  in  describing  the  termi- 
nal utilities  involved.  The  posterior  distribution  of  jl,  with  respect  to 
any  single  MOE,  is  not  used  by  itself  as  a basis  for  any  terminal  action. 
Rather,  this  distribution  is  considered  along  with  the  posterior  distri- 
butions associated  with  many  other  MOE  in  the  overall  assessment  procedure. 
Raiffa  and  Schlaifer  have  suggested  a method  for  handling  this  type  of 
problem  [l8]. 
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If  this  [sample  information]  is  ultimately  to  be  used  in  a 
number  of  second-stage  action  problems,  then  the  immediate 
result  of  the  experiment  will  presumably  be  a posterior  dis- 
tribution of  w [the  variable  of  interest]  in  one  guise  or 
another.  But  how  does  this  help  to  decide  on  sample  size? 

One  possible  approach  is  to  define  some  index  of  "tightness" 
of  the  posterior  distribution  (such  as  the  variance  if  the 
distribution  is  symmetric)  and  then  to  decide  on  a substitu- 
tion rate  between  the  index  of  tightness  and  the  cost  of 
sampling. 

It  is  a solution  procedure  similar  to  this  which  will  be  used  here.  In- 
herent in  this  approach  is  the  idea  of  assigning  an  appropriate  utility 
to  each  value  of  the  index  of  tightness  referred  to  above.  Utility,  as 
used  here,  is  an  expression  of  the  relative  worth  to  the  decision  maker 
of  a particular  value  of  that  index. 


The  Solution  Procedure 

One  logical  choice  for  the  index  of  tightness  to  be  used  in  this 
study  is  the  expected  value  of  the  posterior  standard  deviation.  Assign- 
ing a utility  to  the  values  of  E(  \/  jl"  ) may  be  difficult  in  the  absolute 
sense.  It  may  be  considerably  easier  to  assess  the  utilities  relative  to 
the  total  money  available  for  testing,  say  K^,  For  example,  the  decision 
maker  may  feel  that  an  expected  posterior  standard  deviation  equal  to 
one-half  of  the  prior  standard  deviation  may  be  worth  one-half  of  K^. 

This  last  statement  suggests  the  possibility  of  another  index  of 
tightness,  that  is  the  value  of  s,  where  s is  the  ratio  of  the  expected 
posterior  standard  deviation  to  the  prior  standard  deviation.  This  can 
he  represented  as  a simpler  function  of  n,  VJhich  will  be  important  later, 
and  it  seems  to  be  more  readily  adaptable  to  assessing  utilities.  For 
these  reasons,  it  will  be  used  as  the  index  of  tightness  for  this  study. 
Since  s can  assume  any  value  on  the  interval  (0,l],  it  will  be  necessary 
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to  define  the  utility  of  s using  some  continuous  function,  say  U(s). 

Assume,  for  the  moment,  that  this  is  possible. 

The  cost  of  sampling  may  also  be  thought  of  in  terms  of  utility. 

If  the  cost  of  sampling  is  additive  with  fixed  cost,  K^,  and  variable 

cost,  K , then  the  total  cost  of  sampling,  K , may  be  represented  by 
IT  S 

K = K + K n . (3-1) 

S £ IT 

The  utility  of  the  cost  of  sampling  can  be  expressed  as 

m,)  = - • 

Then  the  utility  of  any  experiment,  e^,  where  n refers  to  the  sample  size 
of  the  experiment,  is  given  by 

U(e^)  = U(s)  - Kg  . (3-2) 

Using  the  optimization  criterion  of  maximizing  utility,  the  solution  to 
the  sample  size  problem  becomes  one  of  finding  the  value  of  n which  maxi- 
mizes equation  (3-2) . What  method  may  be  used  to  find  this  optimum  value 
of  n clearly  depends  on  the  nature  of  U(s).  Several  different  methods 
are  examined  in  the  following  two  sections. 

Linear  Utilities 

In  this  section,  the  problem  will  be  considered  where  U(s)  is 
linear  with  respect  to  s.  In  this  case 
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U(e  ) = as+b  - K . 
' n s 


(3-3) 


Now,  equation  (2-7)  may  be  solved  for  s in  terms  of  n and  n'  yielding 


s = (n')^^^(n'+n)"^/^. 


(3-4) 


Using  the  above  expression  for  s and  equation  (3-1)  for  K , equation 

s 

(3-3)  may  be  written 


U(e^)  = a(n')^'^^(n'+n)"^'^^ 


+b  - K. 


K n 
r 


Differentiating  with  respect  to  n gives 


dU(e^) 

dn 


a(n*)^/^-l/2)(n'+n)‘^/^  - 
- (a/2)(n')^/^(n'+n)"^/^  - K.  . 

A. 


It  does  nut  appear  that  this  can  be  readily  optimized  by  equating 
the  first  derivative  to  zero  and  solving  for  n.  Perhaps  an  easier  way 
would  be  to  use  a nonlinear  one-dimensional  search  such  as  the  golden 
section  search.  To  do  this,  the  objective  function  must  be  unimodal  and 
continuous  [3],  One  way  of  checking  for  these  properties  is  to  investi- 
gate the  convexity,  or  concavity,  of  the  function.  If  the  function  is 
either  convex  or  concave,  the  properties  of  unimodality  and  continuity 
are  guaranteed  [3].  This  will  be  done  by  examining  the  sign  of  the  second 


derivative. 


d U(e  ) t/. 

(a/2)(n')^  (-3/2)(n'+n)"^/^ 

dn^ 

= (3a/4)  (n ')  *+n)  . 

Since  n’  and  n are  both  greater  than  or  equal  to  zero,  the  sign  of 
the  second  derivative  depends  solely  on  the  sign  of  a.  The  value  of  £ 
represents  the  slope  of  U(s).  If  the  slope  is  positive,  the  decision 
maker  is  expressing  a greater  preference  for  larger  values  of  s than  for 
smaller  ones.  This  does  not  seem  to  be  reasonable.  Therefore,  it  is 
felt  that  a can  be  required  to  be  less  than  zero  with  no  loss  of  general- 
ity. With  this  restriction,  the  function  is  concave  and  the  golden  sec- 
tion method  is  applicable.  A computer  program  designed  to  solve  this 
problem  is  contained  in  Appendix  II. 

Power  Function  Utilities 
Now  suppose  that  U(s)  is  of  the  form 

U(s)  = (l-s)'"  Kj.  , (3-5) 


then 

U(e^)  = (l-s)‘'  Kj.  - Kg  . 


Substituting  for  s and  K gives 

s 

U(e^)  = [l  - (n’)^/^(n'+n)’^/^]‘'  K^.  - 


K.  - K n . 
f r 


Differentiating  once  with  respect  to  n gives 


dU(e  ) 
n 

dn 


~ (n')^/^[l-(n')'/^(n'+n)"^/^]‘'"^(n'+n)'^/^  - . 


Once  more  it  appears  that  the  golden  section  method  may  be  the 
best  solution  approach.  Checking  for  concavity  again  involves  taking  the 
second  derivative. 

"^7^  = { [l-(n*)^/^n'+n)”^/^]''  \-3/2)  (n '+n) (3-6) 

+ (n‘+n)’^/^(c-l)[l-(n')^/^n'+n)"^/^]‘'"^-l)(n')^^^ 

X (-l/2)(n'+n)“^/^} 

= (4^)  {(-3/2)[l-(n')^/^(n'+n)"^/^]‘'"^(n'+n)'^/^ 

+ (c-l)(l/2)(n')^/^[l-(n')^/^(n'+n)’^/^]‘'"^(n'+n)"^]-  . 


Determining  the  sign  of  the  second  derivative,  using  equation 
(3-6) , is  not  as  straightforward  as  in  the  case  of  the  linear  utility 
functions.  Clearly,  this  sign  depends  on  the  sign  of  the  parameter  c, 
but  also  it  depends  on  the  relative  magnitudes  of  the  two  terms  inside  the 
brackets.  First,  consider  values  of  c such  that  0 < c S 1,  This  makes 
both  terms  inside  the  brackets  negative  while  the  term  outside  -the  brack- 
ets is  positive,  thereby  making  the  sign  of  the  entire  expression  negative. 


itive,  equation  (3-5)  may  be  rewrit 
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ence  would  be  inconsistent  with  the  objectives  of  the  testing  and, 
therefore,  will  not  be  considered  in  this  study. 

Finally,  for  values  of  c greater  than  one,  the  family  of  utility 
functions  would  be  comprised  of  functions  similar  to  those  shown  in 
Figure  3. 


Figure  3.  Typical  Power  Function  Utilities  for  c > 1 

Functions  such  as  these  could  be  very  useful  since  they  would  allow  the 
decision  maker  to  express  a greater  preference  for  small  values  of  s in 
comparison  to  larger  values.  Unfortunately,  no  general  statement  can  be 
made  as  to  whether  they  are  unimodal  or  not.  This  does  not  mean,  however, 
that  they  cannot  be  optimized.  Bazaraa  and  Shetty  [3]  suggest  that  the 
golden  section  search  may  still  be  used  for  non-unimodal  functions  by 
subdividing  the  interval  of  uncertainty  into  a number  of  smaller  intervals 
and  searching  over  these  smaller  intervals.  The  object  is  to  select  the 


size  of  the  smaller  intervals  so  that  it  is  reasonably  certain  that  the 
function  is  unimodal  over  that  interval.  This  procedure  will  be  illus- 
trated in  the  next  section. 

Illustrating  the  Procedure 

The  solution  procedure  will  now  be  illustrated  using  both  types 
of  utility  functions  described  previously.  Before  doing  this,  however, 
the  computer  program  used  in  the  analysis  will  be  discussed  briefly. 

This  program  is  shown  in  Appendix  II.  It  is  designed  to  perform  a golden 
section  search  using  either  the  linear  or  power  function  utilities.  The 
golden  section  method  determines  the  optimum  value  for  unimodal,  continu- 
ous functions  by  successively  reducing  the  size  of  an  interval  of  uncer- 
tainty [3].  'Cherefore,  it  is  necessary  to  initially  determine  some  inter- 
val of  values  for  the  sample  size  which  is  believed  to  contain  the  optimal 
value.  This  is  done  in  the  program  by  setting  the  lower  bound  of  the  in- 
terval equal  to  zero  ana  the  upper  bound  equal  to  the  largest  sample  size 
allowable  under  the  existing  budget  constraint.  If  the  function  is  uni- 
modal, this  interval  is  then  systematically  reduced  until  it  is  less  than 
or  equal  to  one.  If  the  function  is  not  unimodal,  the  decision  maker  is 
then  required  to  subdivide  the  interval  of  uncertainty  and  then  each  of 
the  smaller  intervals  is  searched. 

The  same  experiment  used  in  Chapter  II  will  be  used  for  this  illus- 
tration. In  order  to  do  this,  however,  several  additional  inputs  are 
necessary,  specifically,  the  budget  constraint,  K^,  the  sampling  costs, 
and  K^,  and  the  utility  function,  U(s). 

To  think  of  a budget  constraint  and  a cost  of  sampling  associated 
with  a single  MOE  may  be  somewhat  unrealistic.  In  practice,  a single 
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experiment  will  produce  data  on  many  different  MOE.  Most  of  the  time, 
the  only  budget  and  cost  figures  associated  with  the  test  are  aggregate 
amounts  in  the  form  depicted  in  Table  3.  Therefore,  rather  than  attempt- 
ing to  determine  the  sampling  cost  for  a specific  MOE  and  the  total  money 
available  for  testing  that  MOE,  it  may  be  much  more  realistic  to  allocate 
to  each  MOE  some  proportion  of  the  aggregate  budget  and  estimated  costs. 
This  is  not-  currently  being  done,  so  it  will  be  necessary  to  approximate 
these  values. 


Table  3.  Total  Cost  Estimates  (Direct  Costs)  [l4] 


Elements  of'  Cost 

Estimated  Cost 
(In  Thousands 
of  Dollars) 

1.  Test  Directorate  Operating  Costs 

19.1 

2.  Player  Participants 

22,1 

3.  Test  Facilities 

30.0 

4.  Items  to  Be  Tested 

.5 

5.'  Data  Collection,  Processing  and  Analysis 

6.4 

6.  Ammunition 

145.4 

7.  Pre-Test  Training 

2.1 

8.  Photographic  Support 

15.0 

9.  Other  Costs 

4.5 

Total 

245.1 

It  is  suggested  that  the  proportion  of  the  aggregate  budget  to  be 
assigned  to  a specific  MOE  be  commensurate  with  that  MOE's  relative 
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importance.  The  OTEA  already  assesses  the  relative  importance  of  MOE  in 
qualitative  terms  [ll].  All  that  is  required  then  is  to  quantify  this 
assessment,  perhaps  through  a series  of  weighting  functions.  It  is  not 
anticipated  that  thia  requirement  would  represent  a major  problem  to  OTEA 
test  design  personnel  who  have  detailed  information  on  the  relationship 
between  the  data  requirements  and  the  operational  issues  being  examined. 

Since  this  type  of  information  is  not  presently  available,  a very 
simplistic  approach  was  taken  to  the  allocation  problem.  Each  of  the  MOE 
was  weighted  equally  in  determining  the  individual  budget  constraint. 

Based  on  an  imposed  test  budget  constraint  of  $250,000.00,  the  individual 
budget  constraint  for  each  MOE,  K^.,  was  derived  to  be  $1,^724.00. 

The  derivation  of  values  for  the  fixed  and  variable  costs  was  ac- 

complished in  a slightly  different  manner.  The  aggregate  estimated  fixed 
cost  was  defined  to  be  the  sum  of  all  those  costs  in  Table  3 except  the 
costs  of  player  participants  and  ammunition.  This  resulted  in  total 
figure  of  $77,600.00.  This  figure  was  then  divided  by  the  length  of  the 
test  in  weeks  to  yield  a fixed  cost  per  week  of  $5,969.00.  Using  this 
weekly  cost  estimate,  each  phase  of  the  test  was  assigned  a fraction  of 

the  total  estimated  fixed  cost  based  on  the  time  required  to  conduct  that 

particular  phase.  The  fixed  cost  associated  with  each  phase  was  then  dis- 
tributed equally  among  the  MOE  being  examined  in  that  phase.  Table  4 pre- 
sents the  results  of  this  process. 

The  variable  costs  are  of  two  types,  those  associated  with  a sample 
size  requirement  for  a certain  number  of  different  individuals  and  those 
associated  with  the  requirement  for  the  expenditure  of  a specified  number 
of  rounds  of  ammunition.  Botti  of  these  variable  costs  were  approximated 
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by  dividing  the  appropriate  total  estimated  cost  figures  presented  in 
Table  4 by  the  total  estimated  requirements  fob  that  resource  [l4].  This 
resulted  in  a variable  cost  for  personnel  of  $57,00  per  week  per  man  and 
a cost  of  ammunition  of  $13.00  per  roilnd. 


Table  4.  Allocation  of  Estimated  Fixed  Costs 


Phase 

Length  of 

Phase 

(weeks) 

Fixed  Cost 
for  Phase 
($) 

No.  MOE 
Examined 

Fixed  Cost 
per  MOE 
($) 

1.  Training 

2 

11,938 

28 

426 

2.  Pilot  Test 

1 

5,969 

0 

0 

3.  Field  Exercise 

3 

17,908 

73 

245 

4.  Live  Fire 

6 

35,815 

36 

995 

5.  Parachute  Delivery 
Demonstration 

1 

5,969 

8 

746 

The  MOE  of  interest  in  this  illustration  is  to  be  examined  during 
the  training  phase  so  the  fixed  cost,  K^,  is  $426.00.  The  test  design 
calls  for  using  the  same  number  of  individuals  throughout  the  training 
phase.  Therefore,  the  variable  cost,  K^,  was  derived  by  multiplying  the 
cost  per  man  per  week  by  the  number  of  weeks  required  to  complete  the 
training  phase  and  then  dividing  the  result  by  the  number  of  MOE  examined 
during  this  phase.  This  process  resulted  in  a value  of  $4.00  for  K^. 

The  above  methods  for  approximating  budget  constraints  and  sampling 
costs  are  . ot  necessarily  being  advocated  for  use  by  OTEA;  they  were  used 
here  to  provide  a starting  point  for  the  demonstration.  This  being  accom- 


40 


plished,  it  remains  to  select  an  appropriate  function  for  U(s). 

The  first  case  to  be  considered  is  that  of  a linear  utility  func- 
tion. The  form  of  this  function  is 


U(s)  = as  + b 


ago 

0 < s g 1 


Consider  Figure  4 below,  by  varying  the  values  of  the  parameters  a and  b, 
it  is  possible  to  represent  U(s)  by  any  negatively  sloped  straight  line 
which  intersects  the  s-axis  between  zero  and  one.  This  provides  the  deci- 
sion maker  with  a rich  family  of  linear  functions  from  which  to  choose. 

The  one  chosen  for  this  illustration  is  the  one  depicted  in  Figure  4. 


Figure  4.  Linear  Utility  Function 

The  equation  for  tnis  function  is 

U(s)  = - K s + K = K (1-s) 
t t t 

Using  this  utility  function  and  the  budget  constraint  and  sampling 
cost  previously  derived,  the  objective  function  becomes 
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U(e^)  = Kjl-(n')~^/^n’+n)"^/^]  - - K^n  . 


This  objective  function  was  used  in  the  computer  analysis,  the  results  of 
which  are  presented  in  Table  5.  The  economically  optimal  sample  size  is 
73.  This  will  reduce  the  magnitude  of  the  prior  standard  deviation  by  a 
factor  of  approximately  four-tenths. 


Table  5.  Computer  Analysis  Using  Linear  Utility 


Lower 

Limit 

Upper 

Limit 

N1 

N2 

U(N1) 

U(N2) 

0.00 

324.50 

123.93 

200.54 

.253 

.055 

0.00 

200.54 

76.61 

123.93 

.314 

."^53 

0.00 

123.93 

47.33 

76.61 

.285 

.314 

47.33 

123.93 

76.61 

94.66 

.314 

.301 

47.33 

94.66 

65.38 

76.61 

.312 

.314 

65.38 

94.66 

76.61 

83.43 

.314' 

.311 

65.38 

83.43 

72.19 

76.61 

.314 

.314 

65.38 

76.61 

69.79 

72.19 

.314 

.314 

69.79 

76.61 

72.19 

74.20 

.314 

.314 

69.79 

74.20 

71.79 

72.19 

.314 

.314 

71.79 

74.20 

72.19 

73.80 

.314 

.314 

71.79 

73.80 

72.19 

73.40 

.314 

.314 

72.19 

73.80 

72.60 

73.40 

.314 

.314 

This  same  analysis 

will  nov7  be 

conducted  using 

two  power 

function 

utilities.  The  first  will  be  defined  by 


U(s)  - (1-s)^/^  Kj. 


0 < s ^ 1 
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Using  this  utility,  the  objective  function  is 

:-(e  ) = (1-s)^/^  K-K.  - Kn  0<ssi 
n t f r 

This  function  was  entered  into  the  computer  program  giving  the  results 
shown  in  Table  6.  As  seen  from  this  table,  the  economically  optimal 
sample  size  is  52.  This  ■ •;  a smaller  sample  size  than  obtained  by  using 
the  linear  utility  function.  This  result  is  to  be  expected  since  this 
power  function  gives  more  weight  to  larger  values  of  s. 


Table  6.  Computer  Analysis  Using  Power  Function  with  c = l/2 

Lower 

Limit 

Upper 

Limit 

N1 

N2 

U(N1) 

U(N2) 

0.00 

324.50 

123.93 

200.54 

.501 

.259 

0.00 

200.54 

76.61 

123.93 

.611 

.501 

0.00 

123.93 

47.33 

76.61 

.631 

.611 

0.00 

76.61 

29.28 

47.33 

.589 

.631 

29.28 

76.61 

47.33 

58.56 

.631 

.631 

47.33 

76.61 

58.56 

65.38 

.631 

.625 

47.33 

65.38 

54.15 

58.56 

.632 

.631 

47.33 

58.56 

51.74 

54.15 

.632 

.632 

47.33 

54.15 

49.73 

51.74 

.632 

.632 

49.73 

54.15 

51.74 

52.14 

.632 

.632 

51.74 

54.15 

52.14 

53.74 

.632 

.632 

51.74 

53.74 

52.14 

53.34 

.632 

.632 

51.74 

53.34 

52.14 

52.94 

.632 

.632 

51.74 

52.94 

52.14 

52.54 

.632 

.632 
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The  second  power  function  utility  to  be  considered  has  the  param- 
eter c equal  to  1.5.  This  function  is  shown  in  Figure  3.  Since  this  par- 
ticular function  is  not  guaranteed  to  be  unimodal  over  all  n,  the  method 
of  subdividing  the  interval  of  uncertainty  into  a number  of  smaller  inter- 
vals was  employed.  The  interval  of  uncertainty,  based  on  the  budget  con- 
straint, is  (0.00,  324.50).  This  interval  was  searched  using  subintervals 
of  length  20.  The  results  are  shown  in  Table  7.  As  can  be  seen  from  this 
table,  the  optimal  sample  size  is  83.  Note  that  the  utility  of  the  ex 
periment  steadily  increases  until  the  optimal  sample  size  is  reached  and 
then  steadily  declines  over  the  remaining  values  of  n.  Thus,  it  is  rea- 
sonably certain  that  a sample  of  size  83  is,  in  fact,  a global  optimal. 


Table  7.  Results  of  Computer  Analysis  Using  Power 
Function  Utility  with  c = 1.5 


Subinterval  Optimal  Sample  Utility  of 

Size  for  Experiment 

Subinterval 


0 

20 

20 

-.144 

20 

- 

40 

40 

.004 

40 

- 

60 

60 

.065 

60 

- 

80 

80 

.083 

80 

100 

83 

.084 

100 

- 

120 

100 

.076 

120 

- 

140 

120 

.053 

140 

160 

140 

.019 

160 

- 

180 

160 

.022 

180 

- 

200 

180 

-.069 

200 

- 

220 

200 

-.121 

220 

240 

220 

-.176 

240 

- 

260 

240 

-.234 

260 

- 

280 

26J 

-.356 

300 

- 

320 

300 

-.420 

CHAPTER  IV 


CONCLUSIONS  AND  RECOMMENDATIONS 

Concltisions 

The  procedures  developed  in  this  study  have  been  structured  around 
Bayesian  inference  and  decision  theory.  The  demonstration  of  these  pro- 
cedures has  been  confined  to  instances  where  the  prior  distributions  have 
been  developed  using  only  objective  prior  data.  This  was  done  to  show 
that  the  procedures  could  be  beneficial  regardless  of  the  strict  limita- 
tions imposed  on  the  use  of  any  subjective  prior  information.  Considered 
in  this  context,  however,  the  analysis  is  not  Bayesian  in  the  purest 
sense,  nor  would  such  an  approach  be  appropriate  when  the  use  of  subjec- 
tive information  is  disallowed. 

Nonetheless,  it  is  felt  that  the  inference  approach  presented  in 
Chapcer  II  represents  a viable  solution  to  the  sample  size  determination 
problem  currently  faced  by  OTEA  test  designers.  Its  major  advantage  over 
the  presently  employed  procedures  is  that  it  has  been  specifically  designed 
to  address  the  problem  of  making  statistically  valid  inferences  regarding 
the  difference  in  two  random  variables. 

The  decision  theoretic  approach,  described  in  Chapter  III,  provides 
the  decision  maker  with  a quantitative  procedure  for  comparing  the  expected 
results  of  an  experiment  to  the  cost  of  that  experiment.  If  the  decision 
maker  is  willing  to  accept  the  concepts  of  utility  theory,  and  to  apply 
them,  this  procedure  will  provide  the  economically  optimal  sample  size. 
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Finally,  should  the  limitations  imposed  on  the  use  of  subjective 
information  be  relaxed,  the  methodology  presented  in  this  study  could  be 
used  in  conducting  a completely  Bayesian  analysis. 

Recommendations 

The  greatest  limitation  to  the  methodology  developed  in  this  study 
is  that  it  is  applicable  only  to  the  case  of  sizing  an  experiment  for  a 
single  MOE.  The  logical  extension  of  this  is  to  the  case  of  multiple 
MOE.  There  are  at  least  two  approaches  to  analyzing  this  case.  One  would 
be  to  apply  multivariate  Bayesian  statistical  theory  combined  with  multi- 
dimensional nonlinear  programming  algorithms.  A second  approach  would  be 
to  view  the  money  required  to  perform  each  of  the  experiments  involved  in 
an  operational  test  as  a capital  investment  and  the  utility  of  each  of 
the  experiments  as  the  return  on  that  investment.  Formulated  in  this 
manner  the  problem  might  be  solved  utilizing  capital  budgeting  techniques. 
If  it  is  possible  to  extend  the  methodology  to  include  multiple  MOE,  then 
it  may  be  possible  to  use  it  in  multifactor  experimental  design  problems. 

Aside  from  extending  the  methodology,  several  other  areas  warrant 
further  investigation.  First,  is  the  assumption  that  the  normal  process 
may  be  used  as  a reasonable  model  for  a large  number  of  operational  test- 
ing problems.  Closely  associated  with  this  would  be  an  investigation  of 
the  variation  in  results  when  the  sampling  process  is  not  normal. 

As  a final  recommendation,  it  is  suggested  that  the  procedures  out- 
lined in  this  study  be  utilized  in  designing  a number  of  operational 
tests  and  that  these  results  be  compared  to  the  results  obtained  using 
the  presently  employed  methods. 


t 
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•APPENDIX  I 


LIGHTWEIGHT  COMPANY  MORTAR  SYSTEM  OT  I TEST 

Gunner's  Examination  Times  [l3] 

DATA 

Test 

System 

81  mm 

LWCMS 

Difference  in 

Participant 

(sec) 

(sec) 

Performance 

1 

358.0 

303.4 

54.6 

2 

367.0 

350.8 

16.2 

3 

299.0 

330.0 

- 31.0 

4 

261.0 

147.5 

113.5 

5 

380.0 

313.0 

67.0 

6 

226.8 

250.0 

- 23.2 

7 

272.0 

247.0 

25.0 

8 

239.8 

273.0 

- 33.2 

9 

235.0 

258.0 

- 23.0 

10 

247.5 

244.8 

2.7 

11 

279.1 

242.7 

36.4 

12 

303.0 

234.2 

68.8 

13 

240.9 

250.7 

- 9.8 

14 

279.0 

296.9 

- 17.9 

oooooooooooooooooooonooooooooooooooo 


APPENDIX  II 


FORTRAN  PROGRAM  FOR  THE  GOLDEN  SECTION  SEARCH  TECHNIQUE 


PROGRAM  MA IN( INPUT, OUTPUT, TAPE 5=INPUT»TAP£6=0UTPUT) 

THIS  PROGRAM  PERFORMS  A NONLINEAR, 

ONE-DIMENSIONAL  SEARCH  USING  THE  GOLDEN 
SECTION  METHOD. 


LIST  OF  VARIABLES. 

AKT  - TOTAL  MONEY  ALLOCATED  TO  TESTING 
THIS  MOE  (BUDGET  CONSTRAINT). 

AKF  - FIXED  COST  OF  TESTING. 

AKR  - VARIABLE  COST  OF  TESTING. 

NP  " WEIGHT  GIVEN  TO  PRIOR  INFORMATION. 

XNP  - TRANSFORM  ON  NP  TO  MAKE  IT  A 

REAL  VARIABLE. 

IP  - INDICATOR  VARIABLE  USED  TO  DESIGNATE 

TYPE  OF  UTILITY  FUNCTION. 

A,B  - SLOPE  AND  INTERCEPT  OF  LINEAR  UTILITY 
FUNCTION. 

C - EXPONENT  OF  POWER  UTILITY  FUNCTION. 

K - COUNTER  DESIGNATING  NUMBER  OF 

ITERATIONS  USED  IN  THE  SEARCH. 

SUBLIM  - LOWER  LIMIT  ON  THE  INTERVAL  OF 
UNCERTAINTY. 

UPLIM  - UPPER  LIMIT  ON  THE  INTERVAL  OF 
UNCERTAINTY. 

XN1,XN2  - POINTS  ON  THE  INTERVAL  OF  UNCERTAINTY 
BEING  EXAMINED. 

UEl  - UTILITY  OF  EXPERIMENT  OF  SIZE 

XNl. 

UE2  - UTILITY  OF  EXPERIMENT  OF  SIZE 

XN2. 

ICHECK  - VARIABLE  USED  TO  CHECK  FOR 
TERMINATION, 


DIMENSION  SUBLIM(20) , UPLIM (20) , UEl ( 20 ) , UE2 ( 20) , 
♦XN1(20) ,XN2(20),VP(2) 

REAL  MP{2) 

INTEGER  IANS,KANS,JANS 

100  FORMATT/Z/^'ENTER  THE  TOTAL  AMOUNT  OF  MONEY  WHICH  */ 
»»YOU  WISH  TO  ALLOCATE  TO  TESTING  THIS  HOE.»/J 

101  FORMAT(///»ENT£R  THE  SET-UP  AND  REPLICATION  COSTS  IN*/ 
♦*  THAT  ORDER.*/) 

102  FORMAT(///*ENTER  WEIGHT  OF  PRIOR  INFORMATION.*/) 

103  FORMATi///*£NTER  TYPE  OF  UTILITY  FUNCTION  TO  BE  USED*/ 
** (ENTER  1 FOR  LINEAR  AND  2 FOR  POWER).*/) 

104  FORMAT(///*ENTER  SLOPE  AND  INTERCEPT,  *, 

**IN  THAT  ORDER*/) 


ooo  ooo  ooo  ooo  oon  ooo 
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105  F0RMATI///*ENrER  EXPONENT.*/) 

107  F0RrtAr<A6) 

200  F0RMAr(I2,5X,F7.2,5X,F7.2,5X,F7.2,5X,F7.2,5X,2F8.3) 
2 01  FORMAT  {///9X,*i.0WER*,6X,*UPPER*/9X,*LIHIT*, 
♦6X,*LIHIT*,9X,*N1*,10X,*N2*»10X,*U{N1)*» 
♦3X,*U(N2)*) 

202  F0RMAT«///*IS  THE  OBJECTIVE  FUNCTION  UNIMOOAL?*/) 

203  FORMAT(///*THE  INTERVAL  OF  UNCERTAINTY  IS  (*» 
*F7.2,^^,*,F7.2»*)*/) 

ZQk  FORMAT C///*ENTER  LOWER  AND  UPPER  BOUNDS*, 

*♦  FOR  SUBINTERVAL.*/) 

205  F0RHATI///*D0  YOU  WISH  TO  SEARCH  ANOTHER  *, 
**SUQINTERVAL?*/) 

DATA  KANS/6HYES  / 

ENTER  THE  BUDGET  CONSTRAINT. 

WRITE(6,100) 

REA0(5,*)  AKT 

ENTER  ESTIMATED  COSTS. 

WRITE(6,101) 

R£A0«5,*)  AKF,AKR 

ENTER  WEIGHT  OF  PRIOR  INFORMATION. 

WRITE(6,102) 

ReA0(5,*)  NP 
XNP=NP 

DETERMINE  TYPE  OF  UTILITY  FUNCTION  TO  BE  USED. 

WRITe{6,103) 

READ(5,*)  IP 

■ READ  UTILITY  FUNCTION  PARAMETERS. 

IFCIP.EQ.2)  GO  TO  10 
WRITE (6, 104) 

REA0(5,*)  A, 8 
GO  TO  20 
10  WRITE<6,105) 

R£A0(5,*>  C 

SET  UP  INITIAL  INTERVAL  OF  UNCERTAINTY. 

20  K=i 

SU8LIM(K)=0 

UPLIM(K)=(AKT-AKF)/AKR 


49 


C 

G 

C 


C 

C 

c 


c 

c 

c 


c 

c 

c 

c 

c 

c 


c 

c 

c 


c 

c 

c 


SEE  IF  THE  FUNCTION  IS  UNIMOOAL. 

WRITE(6»202) 

REAO(5,107J  IANS 
IFilANS.EQ.KANSI  GO  TO ‘501 

DETERMINE  WHAT  SUBINTERVAL  IS  TO  BE  SEARCHED. 

WRITE{6.203)  SUBLIMIK) tUPLIMCK) 

502  K=JL 

WRirE{6»204) 

REAQ(5,»»  SU8LIMiK),UPLIM(K) 

CALCULATE  INITIAL  VALUES  FOR  XNl  AND  XN2. 

501  XNl (K}=. 381924* (UPLIM(K)-SUBLIM(K))+SUBLIM(K) 
XN2{K>=.618*(UPLIM(KJ -SUBLIM^K) »+SUBLIM (K> 

WRITE  COLUMN  HEADINGS. 

WRITE{6,2Q1> 

CALCULATE  UTILITIES. 

920  R1=XNP/(XNP+XN1<K) ) 

S1=SQRT(R1) 

R2=XNP7(XNPfXN2{K) ) 

S2=SQRT1R2) 

IF(IP.EQ.l)  GO  TO  40 

UE1(K)=(1. 0-Sl»**C*AKT-AKF"AKR*XNl(K) 

UE2(K)=(1.0-S2)**C*AKT-AKF-AKR*XN2(K) 

GO  TO  21 

40  U£1U)=A*S1+B-AKF-AKR*XN1<K> 
UE2(K)=A*S2<-B-AKF“AKR*XN2  (K) 

21  IF(XNl(K).LE.XN2tK» ) GO  TO  30 
A1=XN1(K) 

A2=XN2(K) 

XN1(K)=A2 

XN2(K)=A1 

WRITE  RESULTS. 

30  WRITE{6t200>  K.SU8LIH (K) » UPLIM (K) »XN1 (K) , XN2 CK) t 
*U£l(K)»Ue2<K) 

COMPARE  UTILITIES. 

IF(U£1(K>  .LE.U£2(K)  ) GO  TO  934 


oooo  ooo  ooo 
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SET  UP  NEW  INTERVAL  OF  UNCERTAINTY. 

UPLIM(K4-1)=XN2{K> 

SUBLIMCK+1)=SUBLIM(K) 

XN2{K«-i)=XNi(K) 

XNl(Kfl)=UPLIH(K+l)-XN2  tK  + D+SUBLIM  (K+1) 

GO  TO  94^ 

934  UP.LIH(K*1)=UPLIH{K) 

SUBLIM(K+1)=XN1(K) 

XN1(K+1)=XN2(K» 

XN2(K<-ll=UPLIM(K4-l>-XNl(Ktl)+SU8LIH(K*ii 

CHECK  FOR  TERMINATION. 

944  ICHECK=SUBLIM{K)+2 

IFtICHECK.GE.UPLIMtK) .OR.K.GT.20)  60  TO  930 

K=K«-1 

GO  TO  920 

930  CONTINUE 

IF(IANS.EQ.KANS)  60  TO  931 

SEE  IF  ANOTHER  SUBINTERVAL  IS  TO  BE  SEARCHED. 

WRIT£(6,205) 

REAO(5,lfl7>  JANS 
IF(JANS.EQ.KANS)  60  TO  502 

931  CONTINUE 
STOP 
END 
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EXPLANATION  OF  NOTATION 

Chapter  I 

mean  of  normal  density  function 

2 

cr  variance  of  normal  density  function 

X sample  mean 

s^  sample  variance 

f'(0)  prior  distribution  of  0 

f(y|0)  likelihood  function  for  y given  0 

f"(0|y)  posterior  distribution  of  0 

Chapter  II 

fjj^({I,h|m' ,v' ,n' ,v')  normal-gamma  density  function 
h inverse  of 

in',v',n',v'  prior  parameters  for  a normal-gamma  density  function 

(these  are  interpreted  on  page  11) 

m",v",n",v"  posterior  parameters  for  a normal-gamma  density 

function  (these  are  defined  mathematically  on  page  12) 
m,v,n,v  parameters  of  a normal  sampling  distribution  (these 

are  defined  mathematically  on  page 
fg([i,|m,n/vjv)  density  function  for  Student's  t distribution 

p,  expected  value  of  p 

p variance  of  p 

p'  prior  variance  of  p 
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\/T’ 

5' 

vT" 


» 11 

~ll 

d" 


d' 


prior  standard  deviation  of  jl 
prior  mean  value  of  {I 
posterior  standard  deviation 

ratio  of  the  expected  posterior  standard  deviation 
to  the  prior  standard  deviation 

posterior  variance  of  pi 
posterior  mean  value  of  JI 

length  of  a (l-a)  Bayesian  prediction  interval  on 
the  posterior  distribution 

length  of  a (l-a)  Bayesian  prediction  interval  on 
the  prior  distribution 


Chapter  III 

Kj.  total  money  allocated  to  testing 

Kg  total  cost  of  sampling 

fixed  cost  of  sampling 
variable  cost  of  sampling 
U(s)  utility  function  for  s 

U(K  ) utility  function  for  the  cost  of  sampling 

s 

e^  experiment  with  sample  size  of  n 

U(e^)  utility  function  for  an  experiment  of  sample  size  n 
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